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Abstract 


We  present  a theoretical  framework  for  approximation 
techniques  for  nonlinear  system  optimal  control  problems.  Two 
particular  approximation  schemes  that  may  be  used  in  the  context 
of  this  framework  are  discussed  and  typical  numerical  results  for 
two  examples  to  which  we  have  applied  these  schemes  are  given.  We 
conclude  with  a brief  survey  of  related  investigations. 


APPROXIMATION  TECHNIQUES  FOR  CONTROL  SYSTEMS  WITH  DELAYS 


H.  T.  Banks  and  J.  A.  Burns 


1.  Introduction 

In  this  report  a modification  of  earlier  approximation 
schemes  (see  [2, 3, 4, 7, 8])  for  linear  semigroups  is  used  to  develop 
a general  framework  for  approximating  certain  nonlinear  optimal 
control  problems  governed  by  functional  differential  equations 
(FDE) . Particular  types  of  schemes  are  discussed  in  the  context 
of  this  framework,  and  numerical  examples  are  given  that  illustrate 
some  of  our  findings  in  using  these  schemes  on  "typical"  control 
problems.  We  also  present  a short  survey  of  recent  and  ongoing 
efforts  which  entail  the  application  of  semigroup  approximation 
ideas  to  optimal  control  problems  governed  by  FDE. 

V 

Throughout  the  paper,  we  shall  denote  by  L2(a,b)  (or 

V 

L2(a,b)  if  V = 1)  the  customary  Lebesgue  space  of  R -valued  L2 
"functions"  on  (a,b).  The  symbol  I*]  will  be  used  for  all  norms 
and  it  is  to  be  understood  that,  in  general,  |x|  denotes  the  X 
norm  of  x whenever  x is  an  element  of  the  space  X.  The  vector 
space  of  n x m matrices  will  be  written  as  ^ , and 

Z = r"  X L2(-r,0)  will  be  the  usual  product  space.  If  x;  [-r,+“)  -*■  r" 
and  t > 0,  then  the  symbol  x^  denotes  the  function  x^;  [-r,0]  -►  R*^ 
given  by  x^(s)  = x(t+s).  Finally,  we  shall  not,  in  general, 
distinguish  between  row  and  column  vectors  when  the  usage  makes 
clear  our  Intended  meaning. 
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2 . Representation  of  FDE  by  an  abstract  equation  in  Z 

Let  D(t)  G ^ _ be  such  that  the  function  t -*■  |D(t) 
n f in 

is  in  L2(-r,0),  0 = h^  < h^^  < ...  < h^  = r,  and  G 

i = 1,2,.  ..,v.  If  (|) : [-r,0]  -*■  R*^,  then  define  L to  be  the 


operator 


V ,0 

L(<f>)  = I Aj(-h.)  + D/0)(|)  (e)d0. 

i=0  ^ ^ J-r 


We  shall  be  concerned  with  the  nonlinear  control  system 


x(t)  = L(x^)  + f (t,x(t)  ,x^,u(t)  ) , t G [0,tj^] 


(2.1) 


x(0)  = n,  Xq  = 4) , 


where  tj^  is  finite,  (ri,4>)  G Z,  and  f has  the  form 


f(t,y,t,v)  = >#^(t,y,<j')  + {-^(t,y,'p)  + B(t)}v. 


We  assume  that  the  mapping  t ■*  B(t)  G is  continuous,  and 


R^  X Z -*■  r",  R^  X Z = 


(Al)  The  mappings  (t,y,(|')  -►  >^(t,y,ij'),  i = 1,2,  are  continuous. 

(A2)  For  each  ^ C Z bounded,  there  exist  bounded  measurable 

functions  Kj^,K2  such  that 

- >^^(t,x,4>)|  < K.  (t)  I (y,tp)  - (x,<^)| 

for  all  {x,(fr),  (y,<P)  G t G R^,  i = 1,2. 
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(A3)  = 0,  i = 1,2,  and  there  exist  bounded  measurable 

/S  A, 

functions  that  for  t G R, 

I ^j^(t,y,>P)  I < Kj^(t)|(y,ij^)|,  i = 1,2, 

for  all  (y,4')  with  | (y,'J^)  | sufficiently  large. 

For  t > 0,  define  the  operator  S(t);  Z ->  Z by  S(t)  = 

(x(t),x^)  where  x is  the  solution  to  the  homogeneous  linear 
equation  x(t)  = L(x^)  , satisfying  x(0)  = H/  = ip . It  is  known 

that  {S(t)}^>g  is  a linear  Cg  semigroup  on  Z,  and  if  Si/ 
denotes  the  infinitesimal  generator  of  this  semigroup,  then  s/  is 
defined  on  3(si/)  = {(n»4')|  <P  is  absolutely  continuous, 
t G L2(-r,0)  and  n = 4>  (0)  } by  ja^(n»4’)  = (L(<t)),'J).  Let  Z -*■ 

and  Z -►  L2(-r,0)  be  the  coordinate  projections,  TTj^(n,<)')  = n 

and  1^2  ~ • Define  F:  R^  x Z x R*”  ->•  Z by 

F(t,z,v)  = (f  (t,Trj^z,iT2Z»v)  ,0)  . 

Consider  the  abstract  integral  equation  in  Z 
z(t)  = S(t)z  + [ S (t-o)F(a,z (o) ,u(o) )da,  (2.2) 

” J 0 

where  0 < t < t^^  and  z^  = (n»<^)  ^ Z.  If  f satisfies  (Al)-(A3) 
and  u GL2(0,tj^),  then  it  follows  from  the  more  general  results 
established  in  {2]  that  equation  (2.2)  has  a unique  solution  on 


d 
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[0,tj^].  The  following  theorem  is  an  extension  of  the  result  for 
linear  systems  (see  [3])  and  provides  a slight  improvement  on  the 
linear  and  nonlinear  cases  previously  treated  (see  [2]  - Theorem  2,1). 

Theorem  I.  Suppose  that  f satisfies  (A1)-(A3)  and  u G L2(0,tj^). 

If  Zq  = (nf<l>)  is  any  element  in  Z,  then 

z(t;Zj^,u)  = (x  (t;  Zj^,u)  , x^(z^,u)),  (2.3) 

where  t -*•  x(t;zQ,u)  is  the  solution  to  (2.1)  and  t -*  z{tiz^,u) 
is  the  solution  to  (2.2). 

Observe  that  Theorem  I differs  from  Theorem  2.1  in  [2]  in 
that  we  do  not  require  that  z^  = (n/*)*)  belong  to  ^(ja^')  . In 
particular,  the  equivalence  (2.3)  establishes  unequivocally  an 
abstract  formulation  for  the  control  system  (2.1)  as  the  integral 
equation  (2.2). 

The  proof  of  Theorem  I follows  the  proof  of  Theorem  2.1  of 
[2]  once  one  has  established  the  continuity  of  the  mappings 
(Zq,u)  (x(t;z^,u),  x^(z^,u))  and  (Zq,u)  z(t;z^,u)  from 
Z X L2(0,tj^)  into  Z.  Verification  of  these  continuity  require- 
ments entail  only  slight  modifications  of  the  arguments  behind 
Lemma  2 . 2 and  Lemma  2.3  of  [2] . 

RemarJc  2.1:  Theorem  I is  the  foundation  for  the  general  approximation 
technique  to  be  detailed  below.  The  idea  is  to  approximate  the 
linear  semigroup  {S(t)}  which  will  lead,  via  the  equivalence  (2.3), 
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to  approximations  of  the  original  nonlinear  FDR  system  (2.1). 
Before  pursuing  this  matter  we  note  that  Theorem  I is  also  valid 
for  the  somewhat  more  general  nonlinear  systems  considered  in  [2], 


3 . Approximating  control  systems 

We  now  turn  to  development  of  our  abstract  frameworlc  for 
approximating  optimal  control  problems  governed  by  the  nonlinear 
FDR  system  (2.1).  This  frameworlc  extends  the  linear  theory  given 
in  [3]  and  is  a slight  modification  of  the  ideas  developed  in  [2] . 
Let  z(t;zQ,u)  denote  the  solution  to  the  abstract  integral 
equation 


z (t) 


S(t)zQ  + 


t 

S (t-a) F(o,z(o),u(a))da, 

0 


(3.1) 


on  the  interval  [0,tj^]  . Our  goal  is  to  construct  approximations  to 
z(t;zQ,u)  and  we  shall  do  this  by  first  approximating  Z and 
{S(t) }. 

We  say  that  the  sequence  of  quadruples  (t) } , 

N = 1,2,...,  is  an  approximating  sequence  if  the  following  hypotheses 
are  satisfied: 


N 

(HI)  Z is  a finite  dimensional  subspace  of  Z for  each  N. 

MM  N 

(H2)  P : Z -*■  Z are  (continuous)  projections  onto  Z such 

that  limlP*^(n»<l>)  - (n»4>)  | = 0 for  all  (ri»<l>)  € Z. 

(H3)  each  N,  M^:  Z^'*  ->  Z^  is  a linear  operator,  and  there 

is  a sequence  (Ym)  of  constants  such  that  lim  y = 

N-m» 


0 
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and  |M^(J;,0)  - (C,0)  I < Yj^K  I for  all  ^ G r". 

(H4)  The  semigroups  {S^(t)}  satisfy  |s^(t)|  < Ke^^  (where  K 
and  Y are  constants  independent  of  N)  and  if  {r|/4>)  G Z, 
then  limjS  (t)(n,<(>)  - S(t)(n,<l>)|  = 0 and  the  convergence 

N->oo 

is  uniform  in  t on  compact  subsets  of  R^. 


Remark  3.1;  The  definition  of  an  approximating  sequence  given  above 

is  related  to  but  not  the  same  as  one  employed  by  Trotter  in  [15]. 

The  operators  M were  first  introduced  in  [8]  and  they  play  an 

important  role  in  certain  approximation  schemes  (see  the  piecewise 

linear  scheme  in  the  next  section  and  the  discussions  in  [7]). 

N N N N 

If  {Z  ,P  ,M  ,S  (t) } is  an  approximating  sequence,  then 
we  define  for  each  N the  approximating  integral  equation 


N 

z 


N N 

(t)  = s‘'(t)p'^ZQ 


+ 


N N N N 

(t-o)M  P F (o,z  (o) ,u (o) ) do , 

0 


(3.2) 


where  0 < t < t^^  and  z^  = (n»<[>)  G Z.  As  in  [2],  it  can  be  shown 

that  (3.2)  has  a unique  solution  for  each  z^  G Z and  u G L2(0,t^), 

N . 

provided  f satisfies  (Al)-(A3).  Moreover,  since  Z is  finite 
dimensional  it  follows  that  the  integral  equation  (3.2)  is  equivalent 
to  the  ordinary  differential  system 


z^(t)  = A^z^(t)  + M^P^F(t,z^(t) ,u(t)  ) 
z^(0)  = P^Zq, 


(3.3) 


where 


.N 


is  the  infinitesimal  generator  for 


{S^(t)},  i.e.. 
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„N,^,  A^t 
S (t)  = e 

A few  observations  are  now  in  order.  First,  in  some  schemes 

N 

where  elements  of  the  form  (^,0)  are  in  Z (this  is  the  case  for 

the  schemes  to  be  discussed  here  - but  is  not  the  case  in  the  event 

one  considers  the  spline  approximations  in  [6])  we  point  out  that 

N 

F(t,z,v)  = (f  (tfTT^ZjTT^ZjV)  ,0)  is  in  Z . Hence,  in  (3.2)  and  (3.3) 

N N 

it  is  not  necessary  to  project  F(t,z,v)  onto  Z with  P . 
Secondly,  the  idea  of  approximating  FDE  systems  by  finite  dimensional 
ordinary  differential  equations  is  not  new  and  for  a rather  complete 
summary  of  previous  efforts  involving  such  techniques  one  may  see 


[4]  . 

Returning  to  the  problems  under  discussion  here,  we  observe 
that  the  following  two  theorems  may  be  proven  using  the  basic  ideas 
in  [2]  and  [7]  . 


Theorem  II. 
and  suppose 
z*^  (t;zQ,u) 
z (t;z^,u)  - 
and  u G 


Let  {Z*^,P^,M*^,S*^  (t)  } be  an  approximating  sequence 
C L2(0,tj^)  is  bounded.  If  z(t;zQ,u)  and 
are  defined  by  (3.1)  and  (3.2),  respectively,  then 
z(t;z^,u)  as  N uniformly  in  t on  [0,tj^] 


Theorem  III.  Let  {Z^,P^,M^,S^ (t) } 

be  an 

approximating  sequence 

If 

{u*^}  is  a sequence  in  L2(0,tj^) 

and 

{u  } 

converges  weakly 

to 

N N 

u,  then  z (t;zQ,u  ) -*•  z(t;z^,u) 

as 

N -* 

00,  uniformly  in 

on 

I0,t^]  . 

A 
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Remark  3.2;  Theorem  III  is  the  only  result  in  this  paper  that 

does  not  extend  to  the  more  general  nonlinear  systems  treated  in 

[2].  The  critical  requirement  in  establishing  this  convergence  is 

the  weak  continuity  of.  the  mapping  u *■  z(t;zQ,u)  from  L2{0,tj^) 

into  Z.  However,  for  systems  with  nonlinearities  satisfying  (Al)-(A3) 

weak  continuity  can  be  argued  (see  [2]  for  more  comments  on  this  point) . 

N 

If  z(t;ZQ,u)  and  z (t;z^,u)  are  given  by  (3.1)  and  (3.2), 

N rj  N 

respectively,  then  we  let  z (t;zQ,u)  = (x  (t) ,y  (t) ) (where 

x^(t)  £ R^,  y^(t)  G L2(-r,0))  and  observe  that  since 

N 

z(t;zQ,u)  = (x(t),x^)  we  have  convergence  of  x (t)  to  x(t)  in 
R*^  and  y^(t)  to  x^  in  L2(-n,0)  (uniformly  in  t on  [0,t^] 
and  in  u G i^).  Consider  the  optimal  control  problem; 


(^)  Minimize  the  cost 


J (u) 


1 

2 


X (tj^)  Gx  (tj^) 


( s ) Qx ( s ) 


0 


+ u (s)  Ru  (s)  Ids 


subject  to  (2.1)  and  u G where  ^ is  a closed  convex 
subset  of  L2(0,tj^). 


N N N N 

If  {Z  ,P  ,M  ,S  (t) } is  an  approximating  sequence,  then 
we  formulate  the  corresponding  approximating  control  problems: 


(ja^^) 


N 


Minimize  the  cost 


J (u)  = 2 ^ (tj^)Gx  (tj^)  + j 


{x^(s)Qx^(s)  + u(s)Ru(s)}ds 


subject  to  (3.2)  and  u G *2^. 
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Suppose  now  that  for  each  N,  iT  is  an  optimal  control  for 
the  approximating  problem  (note  that  need  not  be  unique 

in  the  general  case  of  problems  with  a nonlinear  system) . Employing 
standard  arguments  (see  Theorem  4.1  of  [3])  one  can  show  that  {u^} 
is  a bounded  sequence  in  L^(0,t^)  and  possesses  a subsequence 
which  converges  weakly  to  a control  that  yields  a solution  to  problem 
( . Indeed,  one  finds 

— N 

Theorem  IV.  Suppose  that  u is  an  optimal  control  for  problem 

N = 1,2,...  . Then  there  is  an  optimal  control  u*  for 

-^k  -^k 

problem  (^)  and  a subsequence  {u  } such  that  {u  } converges 
to  u*  and  x ^(t;Zj^,u  x(t;ZQ,u*)  as  Nj^  «.  Moreover, 

if  u*  is  the  unique  solution  for  (,^)  then  the  original  sequence 
{u  } converges  to  u and  x (t;zQ,u  ) x(t;Zj^,u  ),  uniformly 
in  t. 

4 . Two  approximation  schemes 

In  this  section,  we  outline  two  particular  schemes  that  are 
included  in  the  general  framework  presented  in  Section  3 and  discuss 
some  numerical  examples  based  on  these  schemes.  In  order  to  facilitate 
exposition  in  illustrating  these  ideas,  we  consider  the  simplest  case 
where  the  linear  part  of  equation  (2.1)  has  the  form  L(i}>)  = Aq<|)  (0)  + 
(-r)  . 

Let  {tj},  j = 0,1,2,. ..,N  be  the  partition  of  [-r,0] 
defined  by  t^  = — ^ and  let  t^  be  the  midpoint  of  f'-j'^j-l^  “ 

We  let  Xj  denote  the  characteristic  function  for  [tj,tj_j^),  while 
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4'j(t)  = (t-tj)Xj(t).  If  (ri,<<>)  G Z,  then  for  j = we 

define 


jN  ^ N 

j r 


_ 12N 
^ j 3 

■J  V 


(s) ds , 


N 


1 ; 
3 


(s-t^)4)  (s)ds, 
N J 


and  we  define  <t>Q  to  be  n. 

The  approximating  sequence  for  what  we  shall  call  the  averaging 
approximations  is  defined  by  quadruples 


AVE  = {z^,P^,M^,s'^(t)  } where: 


z""  = n G r",  0 = I v^x^,  v"  gr"}; 

j=l  1 3 J 

P^(n,<f>)  = (n,  I 4>^X4)  ; 

j=i  J ^ 


the  identity  on 
N,.  . At 

S (t)  = e , where  A is  the  operator 


A^(n,^)  = (AqH  + F ^^^1  - 


The  operator  A^  may  be  loosely  described  as  "the  approximation 

N 

of  the  infinitesimal  generator  Ji/  for  {S(t)}  on  the  subspace  Z " 
by  using  forward  differences  to  approximate  <<>  (see  [3],  [4]). 


The 
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proof  that  AVE  is  an  approximating  sequence  may  be  found  in  [4] . 

The  approximating  sequence  for  what  we  have  termed  piecewise 
linear  approximations  is  given  by  the  quadruples 


aM  ysM  /nM 

PWL  = {Z  ,P  ,M  ,S  (t)}  where: 

SN  r,  A \ I ^ A r N N . N,„N  N N n, 

z = {(n,4>)|  n e R , <t>  = I v-x-  + w.^.,  v.  ,w.  gr  }; 

j=l  J J J J j J 
N 


p = (n,  i <<> -x^  + ; 

j=i  1 : 3 J 


wN  , r IN  IN  , N,„1N,  , , , 

M (n,  L V-X-  + w.H'.)  = (n,g)  f where 
j=l  J 1 J J 

rl  ^ N,  N ^ ,N  . N,,„N  r N N . N,„Ni 

g = {j  n + + ^7  h + 

-N  A^t 

S (t)  = e 


N 


N N . N.„N> 


Here  A is  the  operator  defined  (again  this  involves  a forward 

• ^ N 

difference  approximation  for  <j>)  by  A (ri»<}>)  = where 

N 


5 = A 


qH  + ^ and  (t)  = I is  determined  by 

j=l  ^ ^ ^ ^ 


pN . E {A^n  t A,(^;;  - 


3 


N /aN  aN.  qN  N ,^;N 

? '♦j-l  - ♦}>'  ®j  ' F '♦j-i  - ♦!>' 


The  proof  that  PWL  is  an  approximating  sequence  is  given  in  [7] 
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We  present  next  a sununary  of  two  excunples  that  are  typical 
of  the  behavior  we  have  observed  in  using  the  above  schemes  in 
numerical  computations.  For  results  of  tests  with  numerous  other 
examples  (both  linear  and  nonlinear)  one  should  consult  [2] , [4] , 

[5],  [7],  [8). 


Example  1 ; We  consider  the  problem  of  minimizing 


2 2 1 
J(u)  = j x(2)^  + i 


over  u E = L2(0,2)  subject  to 


[u(t) ]^dt 


x(t)  = x(t-l)  + u(t),  0 < t < 2, 

X (0  ) = ({)  (0  ) , -1  < 0 < 0 , 


where 


4>  (0) 


■{ 


20 


-20  + 1 


-1  < 0 < - J , 

■ ^ < 0 < G 


One  can  use  necessary  conditions  for  delay  system  control  problems 
to  find  an  analytical  solution  to  this  problem  (for  details,  see 
I7J).  The  optimal  control  is  given  by 
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0 < t < 1, 

1 < t < 2, 

where  6 ••  1.05102,  and  the  corresponding  value  of  the  payoff  is 
J(u*)  2.55531.  In  Figure  1,  we  plot  typical  approximate  controls 

(for  N = 16)  for  the  AVE  and  PWL  schemes  described  above,  along 
with  a plot  of  u*. 

Example  2 ; The  problem  is  to  minimize 

2 

J(u)  = J x{2)^  + I I {x(t)^  + u(t)^}dt 

over  u G ^ = L2(0,2)  subject  to  the  nonlinear  system 

x(t)  = x(t)sin  x(t)  + x(t-l)  + u(t),  0 < t < 2, 

x(0)  =10,  -1  1 0 1 0- 

While  we  are  not  able  to  solve  this  problem  analytically,  we  can 

use  directly  the  necessary  conditions  for  delay  systems  to  get  a 

numerical  approximation  for  u*  (see  [2]).  This  can,  in  turn,  be 

used  to  check  the  convergence  of  schemes  such  as  those  under 

—8 

discussion  here.  Figure  2 contains  plots  of  u (using  the 

AVE  scheme)  and  the  independently  obtained  numerical  approximation 

for  u*. 

5.  Final  remarks. 


We  conclude  with  brief  comments  on  recent  efforts  directly 
related  to  those  reported  here.  First,  complete  and  detailed 
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discussions  (along  with  error  estimates,  numerical  results  for  a 
number  of  other  examples,  and  remarks  on  the  role  of  the  operators 
M ) for  the  averaging  ( AVE)  and  piecewise  linear  (PWL)  schemes 
detailed  above  may  be  found  in  [4],  [7],  and  [8].  Other  recent 

contributions  involving  the  use  of  a semigroup  framework  for  approxi- 
mation methods  include  the  work  of  Reber  in  [11]  and  Banks  and 
Kappel  in  [6].  Reber,  employing  factor  space  methods  (see  Chapter  5 
of  [10])  has  carried  out  theoretical  and  numerical  investigations 
for  schemes  that  entail  simultaneous  discretizations  (in  both  the 
time  and  state  variables) . This  underlying  idea,  which  leads 
directly  to  finite  dimensional  difference  equation  control  problems 
(as  opposed  to  the  ordinary  differential  equation  approximating 
problems  such  as  those  discussed  above) , was  used  in  [11]  to  develop 
a theoretical  framework  for  problems  with  general  nonautonomous  linear 
FDE  control  systems.  In  [6]  the  authors  show  that  the  framework 
developed  in  [2],  [3],  and  [4]  is  an  appropriate  setting  for  the 

use  of  splines  in  approximating  linear  and  nonlinear  FDE. 

Several  other  investigators  have  used  semigroups  to  treat 
nonlinear  FDE  in  the  spirit  of  the  framework  given  in  [3].  Reddien 
and  Webb  [12]  and  Sasai  and  Ishigaki  [13] , assuming  a global 
Lipschitz  condition  on  the  nonlinearities  of  the  system,  use  ideas 
from  nonlinear  semigroup  theory  (as  developed  in  recent  years  in  the 
investigation  of  nonlinear  evolution  equations  in  Hilbert  spaces) 
to  obtain  convergence  results  for  approximation  schemes  of  the 
averaging  type  discussed  above.  Kappel  and  Schappacher  [9]  develop 
a "local"  nonlinear  semigroup  theory  which  requires  only  a local 
Lipschitz  condition  on  their  autonomous  nonlinear  FDE  and  obtain 
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approximation  techniques  which  are  also  of  the  averaging  type.  To 
date,  only  Sasai  has  considered  any  of  these  nonlinear  semigroup 
results  in  the  context  of  optimal  control  problems  and  the 
theoretical  results  given  in  [14]  appear  to  be  applicable  in 
practice  only  to  a much  more  restricted  class  of  control  problems 
(problems  with  admissible  sets  of  smooth  controls  that  are  essentially 
conditionally  compact)  than  those  discussed  above  or  in  [2] . 
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